An Interactive Talking Electronic Cookbook 



Related Applications 

The present application is a continuation-in-part of U.S. Patent Application 
Serial No. 09500, 599, filed February 10, 2000 (pending), and claims all rights of 
priority thereto. 

Field of the Invention 

The present invention generally relates to an apparatus facilitating the 
cooking process for a home chef. More specifically, the present invention relates to 
an apparatus which uses spoken instructions to prompt and direct the cook and 
understands a set of specific spoken commands pertinent to the food preparation 
and recipe organization processes. Additionally, the present invention allows 
recipes to be input via a hand-held scanner, Internet download, a keyboard or any 
other input device known in the art; and further allows these recipes and shopping 
lists created based on these recipes to be shipped to other recipients. 

Background of the Invention 

According to the present practice, one of the most popular methods of 
presenting information to a cook is through a recipe book, i.e. cookbook. Ordinary 
cookbooks, however, are often awkward to use while engaged in food preparation. 
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Recently a variety of electronic cookbooks were developed to facilitate the cooking 
process. 

One exannple of the electronic cookbook is presented in U.S. Patent No. 
5,836,446, the disclosure of which is fully incorporated herein by reference. The 
'446 patent discloses a connputerized electronic cooking encyclopedia intended to 
provide information about preparation of a worldwide variety of foods and 
beverages. The software provides a hierarchical structure whereby the user can 
branch through multiple paths to access cultural information, menu category, 
ingredients and preparation methods. When the cooking encyclopedia of the '446 
patent is connected to an audio enabled database, it is capable of giving vocal 
instruction to the cook. The cook, however, can not interact with the cookbook. 
For example, the cook can not issue vocal commands to the book, can not vocally 
inform the book about the number of portions he/she intends to cook, can not turn 
the book "on" or "off" vocally, etc. Additionally, the cook can not add recipes to 
the cookbook disclosed by the '446 patent. Each recipe in the disclosed 
encyclopedia has to be hierarchically formatted and connected to all complicated 
features of the book. An ordinary cook, without any computer programming 
experience, can not do it. 
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Thus there is a substantial interest in the art for an electronic cookbook 
which can interact with a cook vocally, which allows a cook to add recipes to the 
database and which is sufficiently sinnple for a cook to operate. 

Summary of the Invention 

It is an object of the present invention to provide an apparatus for facilitating 
food preparation which will allow the user to interact with the apparatus by vocally 
communicating user's commands and by receiving spoken cooking instruction. 

It is another object of the present invention to provide an interactive 
apparatus for facilitating food preparation which will allow the user to load 
additional recipes into the database of recipes already stored on the apparatus. 

Other objects, advantages and features of this invention will be more 
apparent hereinafter. 

The device accomplishing the above enumerated objects is an interactive 
electronic cookbook which uses spoken instructions to prompt and direct the cook 
during food preparation and understands a set of specific vocal commands pertinent 
to the food preparation and recipe organization processes. Additionally, the present 
invention allows recipes to be input via a hand-held scanner, Internet download, a 




keyboard or any other input device known in the art; and further allows these 
recipes and shopping lists created based on these recipes to be shipped to other 
recipients electronically. 

Brief Description of the Drawings 

A full understanding of the invention can be gained from the following 
description of the preferred embodiment when read in conjunction with the 
accompanying drawings in which: 

Figure 1 is a flow diagram depicting the available functions of the cookbook 
and how they can be accessed by the user; 

Figure 2 is a flow diagram showing interaction between different components 
of the cookbook; 

Figure 3 is a table of files and their associated fields which are included into 
a data base of the preferred embodiment of the present invention; 

Figure 4 is a table of fields within the Recipe Part file and their assigned 
values corresponding to a representative recipe; 

Figure 5 is a table of fields within the Recipe Assembly file and their assigned 
values corresponding to the representative recipe; 
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Figure 6 is a table of fields within the Recipe Ingredients file and their 
assigned values corresponding to the representative recipe; and 

Figure 7 is a table of fields within the Recipe Header file and their assigned 
values corresponding to the representative recipe. 

Detailed Description of the Preferred Embodiment and the Drawing 

In accordance with the preferred embodiment of the present invention, an 
electronic cookbook is provided which, as shown in Fig. 2, has a memory storage 
device 100 containing a hierarchically structured database of cooking recipes. Any 
known memory storage device may be used in connection with the present 
invention, such as a hard drive, CD-ROM, zip drive, etc. Loaded, formatted and 
edited recipes may be added to the database, as shown in step 102, and may be 
later selected by a user, as in step 104, to be either displayed on a screen 108, 
preferably provided with the cookbook, step 106, or transferred to the text-to- 
speech software, step 1 10, to be converted into spoken language and 
communicated to the user over a provided speaker 1 12, step 1 14. Recipes may be 
added to the database through different input devices 1 1 6, such as a keyboard or a 
scanner, as in step 120, or orally, steps 122 and 124, through a microphone 1 18 
and voice recognition software 126. In the preferred embodiment of the present 
invention the ViaVoice voice recognition and text-to-speech application is used. 




ViaVoice is produced by and may be obtained from the IBM Corporation. However, 
any other suitable voice recognition or text-to-speech software may be used in 
connection with the present invention, for example DragonDictate, currently 
manufactured by Dragon Systems. 

Shown in Fig. 3 is the design of the data-base of recipes used in the 
preferred embodiment of the present invention. The following files should 
preferably be included for each recipe within the data-base: Recipe Header, Recipe 
Part, Recipe Ingredient, Recipe Assembly, Ingredient, Measure Preparations 
(individual). Preparations (linked list of individual preparations). Course (linked list). 



The above files will be more particularly described with regard to the 



following representative recipe: 


Bruce's Variation on Odd Flavor Chicken 


TIME: 1 hour 


Ingredients 




1 chicken, 2 to 3 pounds 




1 head lettuce 




INGREDIENTS for SAUCE, GROUP A 




4 tablespoons soy sauce 
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2 tablespoons honey 

1 clove crushed garlic 
72 teaspoon salt 

INGREDIENTS for SAUCE, GROUP B 

3 tablespoons peanut or corn oil 

2 scallions, chopped 

4 slices fresh ginger root, minced 

72 teaspoon Szechuen peppercorn, slightly crushed 
% teaspoon crushed red pepper 



Preparation and Cooking 



Wash and clean the chicken. Bring 3 quarts of water to boil in a large, deep pan. 
Submerge the chicken in the boiling water for 1 5 minutes. Turn off the flame and let 
the chicken cool in the water for at least 20 minutes before taking it out. Remove all 
skin and bones from chicken. Break into bite sized pieces and place in refrigerator. 
Wash, shred and arrange the lettuce leaves on a large platter. Arrange the chicken over 
the lettuce. Pour the warm sauce (see below) over the chicken and serve. 
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To prepare the sauce, combine the ingredients in Group A in a bowl and let stand 
for at least 5 minutes. In a small pan over a low flame heat the Group B ingredients 
for about 3 minutes. Then pour B into the bowl containing A. Mix well. The sauce is 
now ready to pour over the chicken. 

Tips 

This dish can be prepared beforehand. The chicken can be cooked the day before, 
cleaned and cut an hour or so before dinnertime, arranged over lettuce on the platter 
and left in the refrigerator. The two groups of ingredients for the sauce can be 
assembled and placed in a bowl and saucepan respectively. The final preparation takes 
but a few minutes. Since this dish is served cold, it is especially good for summer 
days. 

The original Chinese name is Odd Flavor Chicken, because it uses such a variety 
of ingredients. 



The Recipe Header file will preferably have the following data fields: recipe 
number, a five digit number assigned by the system; title, text field containing up to 
80 characters naming the recipe; course, a two-character abbreviation obtained from 
the Course file; parts, a one-digit number indicating number of parts to the recipe; 
servings, a two-digit number quantifying the number of portions served; and 
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scalability, a boolean indicator indicating whether ingredients are scalable. Shown in 
Fig. 7 is the Recipe Header file having its data fields identified with respect to the 
recipe example given above. The system assigned number 1 2345 to the recipe, this 
number will serve as the recipe number identifying this particular recipe within the data 
structure. The title of the recipe is stored as a text in the title field. Abbreviation EN 
is used to characterize the recipe as an entree in the course field of the data-base. 
There are 3 parts of the recipe and it is intended for 4 servings. These two numbers 
are stored as the parts and servings fields of the data structure, as shown in Fig. 7. 
The boolean indicator Y is used to indicate that the recipe is scalable, i.e., it may be 
recalculated for a different number of servings. 

The Recipe Part file will preferably have the following data fields: recipe number, 
a five digit number assigned by the system; part, a one-digit number indicating the part 
of the recipe; part name (if there are more than one part), a description of the part 
having up to thirty characters; servings, a two-digit number quantifying servings or 
measure; measure, weight or volume from the Measure file; and notes (optional), text 
item of variable length to assist a chef. As shown in Fig. 4 with respect to the above 
recipe example, the Recipe Part file adds the following information to the data 
structure: the recipe has three parts (the part field); their names are "Main", "Sauce 
Group A" and "Sauce Group B" (the part name field), and notes helpful to a cook (the 
notes field). Data-base fields which are common for several files should have the same 
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value assigned to them throughout the data structure of the same recipe. For 
example, the servings element for recipe 1 2345 will always be 4 regardless of the file 
it is used in. 

The Recipe Ingredients file will preferably comprise the following data fields: recipe 
number; part; ingredient number, a five-digit number from an Ingredient file; amount, 
a number indicating quantity of the ingredient as a range; measure, weight or volume 
from a Measure file; and principal, a boolean indicator of whether the ingredient is a 
principal one. The Recipe Ingredients file may also optionally include the following 
fields: size, a number indicating the size of an item as a range; units, a descriptor 
associated with some ingredient types (e.g., clove, head, spring, pinch, bunch, etc.); 
preparation, a pointer to a linked list of preparation types; modifier (e.g., small, 
medium, large, jumbo, etc.); container, indicating the type of container (e.g., can(s), 
jar(s), box, etc.); comment, a text item having up to thirty characters; and alternate, 
a pointer to an alternative ingredient record. As shown in Fig. 6, several ingredients 
are identified in the ingredient field of the data structure for the above recipe. In the 
preferred embodiment of the invention, the names of the ingredients are coded as five- 
digit numbers. Generally, with respect to Fig. 6, all entries which appear italicized are 
preferably stored in different files of the data-base in coded form and are shown in the 
figure in uncoded form for the sake of clarity. Each of the ingredients has a 
corresponding amount listed in the amount field. In order to properly communicate the 
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recipe, the amount field should be read in conjunction with the measure field and the 
units field. For example, the ingredient "garlic" should be used for the above recipe 
in the amount of 1 (the amount field) clove (the unit field), and the ingredient "peanut 
oil" should be used in the amount of 3 (the amount field) tablespoons (the measure 
field). Some ingredients, for example dried red pepper, require a preliminary individual 
preparation (it has to be "crushed") which is indicated in the preparation field. 

The Recipe Assembly file preferably has the following fields: recipe number; part; 
sequence number, a two-digit number generated by the system; and instruction, a text 
item ending with a period. As shown in Fig. 5, the text of the "Preparation and 
Cooking" section of the above recipe is divided into separate recipe preparation steps, 
and then each of the steps is entered into the data-base as a textual field with a 
corresponding sequence number assigned by the system. There is a different 
sequence for each part of the recipe. For example, step number one for the first part 
of the above recipe is to "wash and clean the chicken" and step number one for the 
third part of the recipe is to "... heat the Group B ingredients for about 3 minutes." 

In addition to the above files, the data structure will preferably contain several 

auxiliary files used, for example, for storing the coded information. There are several 

of these files, for example the ingredient file, the measure file and the individual 

preparation file. The Ingredient file should preferably have the following data elements: 
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ingredient code, a five-digit number assigned by the system; description, a textual 
description of the ingredient; food type (e.g., grain, spice, fruit, vegetable, fish, etc.); 
and, optionally, nutritional values, currently provided by the Department of Agriculture. 
The Measure file will preferably have a measure code and description elements. The 
individual Preparation file will preferably only have a preparation element taken from 
the list of possible preparation processes, such as shredded, chopped, minced, 
washed, boiled, etc. The Preparation's linked list file will, in addition to the preparation 
element, have two pointer elements, a pointer to the next preparation and a pointer to 
the previous preparation. The Course file may have two elements: a course code, 
which is a two-character code describing a course type, and a course description, for 
example, soup, appetizer, entree, etc. The cookbook may be made programmable, 
allowing each user to include additional files and/or data elements in accordance with 
the user's needs. 

In accordance with the present invention, when the apparatus is powered on a 
menu of options appears, each of which can be chosen by a user by means of 
keyboard, mouse or voice. As shown in Fig. 1, the following options are preferably 
available for the user: SELECT, TIME, EDIT, GET RECIPE, RULES, LOAD RECIPE, 
MOVE RECIPE, GLOSSARY, SHOP and STOP. SELECT, block 10, allows a cook to 
select a recipe from those available in the memory. TIME, block 20, enables the 
naming and setting of time. A plurality of different names and timers may be set. 
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COOK, block 40, prompts the cook vocally through the instructions of the selected 
recipe. EDIT, block 30, permits the creation of new and the alteration of existing 
recipes. GET RECIPE, block 31, enables the cook to prepare text for loading as a 
recipe from a scanner, a sequential file on a diskette or CD-ROM, or allows 
downloading of recipe text from the Internet. LOAD RECIPE, block 32, is the process 
by which a new recipe to be added to the cookbook from a sequential file (e.g., text 
file) is parsed interactively using the cook to resolve ambiguities, and then is tested 
and accepted into the cookbook. RULES, block 33, permits the cook to choose 
whether to use metric or imperial ingredient measure and to change the default order 
of recipes in the SELECT option. MOVE RECIPE, block 34, provides for copying 
recipes between media and for recipe removal. SHOP, block 35, enables the cook to 
edit, print and transmit a shopping list to a supermarket. GLOSSARY, block 36, allows 
the user to choose a term for which the system can read aloud an explanation. Finally, 
STOP closes all files and powers down the cookbook. It is to be understood that the 
invention may be provided with foreign language software. The commands described 
in the present application will then be translated into the appropriate language. 

The SELECT option displays all recipes in the cookbook's database arranged in the 
default hierarchical order, selected and changeable under RULES. Navigation through 
the recipes is preferably done by voice input; however, keyboard or mouse may also 
be used. The following set of commands may be implemented on the apparatus: 
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PAGE-UP, PAGE-DOWN, UP-LINE, DOWN-LINE, TOP, BOTTOM, MIDDLE and LETTER 
A through LETTER Z. Any other simple navigational command may be programmed 
into the apparatus and recognized by the voice recognition software. Once the desired 
recipe is highlighted, the cook may display it, block 37, begin cooking using the 
highlighted menu, edit it, send it to a selected destination, block 38, for example via 
e-mail or fax, set a timer, print it, block 39, create a list of ingredients required for the 
selected recipe, block 41 , or return to the START. See, Fig. 1 . Each recipe preferably 
has an ingredients section, listing all the ingredients and their quantities, an assembly 
section, giving step by step instructions to the cook, and a notes section, giving some 
suggestions and/or recommendations to the cook. Any or all of the above sections may 
be spoken vocally to the cook by the cookbook. 

The TIME option allows the cook to set as many timers as are needed. When 
setting a timer, the user is preferably prompted vocally for a dish item name for which 
the timer is being set up. The name is recorded and will be associated with the timer 
until the timer expires. The user is then prompted to input time, usually in minutes or 
hours and minutes. Depending on the vocabulary size of the voice recognition 
software, amounts may be specified as usual (e.g. twenty seven) or as several single 
digits (e.g. two, seven). When a timer expires, the bell is rung, the message is 
displayed on the screen, and the specific timer name is vocally announced. 
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EDITing occurs to recipes already loaded in the cookbook, or to new recipes that 
must conform to the structure of the cookbook. The input for this process is 
preferably via a keyboard. This option allows the cook to test the entered or altered 
recipe by having the recipe instructions read aloud. When recipes are newly entered, 
ingredients are chosen from a standard list, obtained from the U.S. Department of 
Agriculture's Food Description File and extendible by the cook when new or unlisted 
ingredients are encountered. The ingredients should preferably be standardized and 
comparable with supermarket names, so that the list of ingredients could be sent out 
to a supermarket for shopping. 

The RULES option allows the cook to select two defaults: the measure and the 
order of the recipes on the selection screen. The cook may select metric or non-metric 
measurements system. If a particular recipe was entered in one system it may be 
converted into the other at the cook's request. There are three preferred orders for the 
recipes to be displayed on the selection screen: alphabetically by recipe name, 
alphabetically by recipe name within course, or alphabetically by recipe name within 
the principal ingredient. The cook may switch the default recipe order while in the 
process of searching through the recipes on the selection screen by using the 
command RULES. It is not necessary to return to START and then access RULES. 
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The COOK option preferably prompts the cook vocally through the selected recipe. 
The cookbook interacts with the cook by first asking to identify the number of portions 
that the recipe is intended for. If the recipe was entered as scalable (for the different 
number of portions), the cookbook will recalculate the ingredients and present to the 
cook a finished version of the recipe scaled for the desired number of portions. There 
is preferably a small set of voice commands which allow the assembly and notes 
sections of the selected recipe to be spoken to the cook when he/she is ready for 
them. The commands included in (but not limited to) the preferred embodiment are: 
Next Instruction; Last Instruction; Repeat; Select Another (recipe); Other Recipe, to 
repeat the last instruction from the other recipe; Set Time, to set a timer; Tell Time, 
to indicate time remaining on all timers; Read Note, to read the recipe notes; and 
Return to Start. All of the above commands may be entered from the keyboard, if 
desired. 

GET RECIPE is a preparatory option, which allows the cook to input the recipe into 
the cookbook. In order to be input, the recipe may be scanned by an attached optional 
line scanner, downloaded from the Internet or another storage medium, copied from 
a diskette, or entered manually from a keyboard. Depending on the amount of 
available memory and the quality of the voice recognition software, recipes may be 
entered vocally. All recipes, regardless of the type of input, are entered as sequential 
files, which can subsequently be used to structure the text, with the user's assistance, 
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for inclusion as a recipe in the cookbook. As shown in Fig. 1 , after a recipe has been 
created as a sequential file, the LOAD RECIPE process is automatically invoked. The 
LOAD RECIPE option fornnats the text of the input recipe to conform to the format of 
the cookbook. Depending on how the externally originated recipe was initially 
formatted it might be easy or difficult for the program to discern and assign values to 
the various ingredients. The cook is enlisted to identify, and possibly reenter, parts of 
the recipe that the program can not discern. The program is also designed to 
automatically substitute and insert quantities from the ingredients section into the 
assembly section at the first mention of an ingredient, so that the cook does not have 
to check back at the ingredient list to know how much of the ingredient to use. The 
program then loads the entire recipe in a temporary form, showing the recipe on the 
screen. The cook may visually review the recipe and/or test it by making the cookbook 
recite the instructions vocally. If the recipe tests satisfactorily, the cook accepts it into 
the cookbook and the sequential file is deleted. In addition to recipes themselves, the 
nutritional information about a particular ingredient or a particular recipe may be stored 
in the cookbook and supplied to the cook at his/her request. Such nutritional 
information is available from the Department of Agriculture for most of the commonly 
used ingredients. 

The MOVE RECIPE option allows for a variety of movements of recipes. A recipe 
can be duplicated using another name and then edited to provide uniqueness. Some 
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or all of the recipes in the cookbook can be copied to a disk, a writable CD or a 
magnetic tape. Recipes already in the cookbook's format may be copied from a disk, 
CD-ROM, DVD or any other storage media directly into the cookbook, thereby avoiding 
the GET RECIPE and the LOAD RECIPE processes. MOVE RECIPE also allows for 
removal of any recipe from the cookbook if desired. 

The PRINT option allows the cook to print any selected or all recipes in the 
cookbook. Any individual recipe may be printed as written or as a version scaled for 
the required number of portions. The cook may print an ingredients list for the 
selected recipe(s). Related to the PRINT option is the SHOP option, which allows the 
cook to create and edit the shopping list from the list of necessary ingredients for all 
selected recipes. The shopping list can be printed or sent electronically, e.g. via e- 
mail, to a supermarket or an on-line grocery store for fulfillment. 

The GLOSSARY option permits the cook, using the same set of voice navigational 
commands available in the SELECT option, to read or, if desired, listen to a description 
of cooking terms with which the cook is unfamiliar. 

The present invention may be executed on a wide variety of electronic carriers. 
For example, the electronic cookbook may be implemented as an independent 
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electronic handheld device or designed as an integral part of a household appliance. 
It can also be used as a standalone computer application or integrated into a website 
on the Internet. To facilitate the vocal interaction between the cook and the 
cookbook, the device may be provided with a wireless microphone. If the cookbook 
is constructed as a handheld electronic device, it may be provided with a serial or USB 
port to enable network capability. The network connection may also be wireless. 
Connecting several such cookbooks into a network will allow their users to exchange 
recipes without using any of the above mentioned input devices. The same serial, USB 
or wireless port may also be used to connect the device to a personal computer. 

While the invention has been shown and described in its preferred embodiment, 
it should be appreciated that the invention may take many other forms without 
departing from the scope of the invention taught by the inventor. The scope of the 
claims, as interpreted by the Courts, both literally and with due deference to the 
doctrine of equivalents defines the scope of protection to which the inventor is 
entitled. 
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